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INTRODUCTION 


Analysis of student-level data to inform policy and promote student success is a core function of executive 
higher education agencies. Postsecondary data systems have expanded their collection of data elements 
for use by policymakers, institutional staff, and the general public. State coordinating and governing boards 
use these data systems for strategic planning, to allocate funding, establish performance metrics, evaluate 
academic programs, and inform students and their families. The State Higher Education Executive Officers 
association (SHEEO), as part of a project funded by the Bill & Melinda Gates Foundation (BMGF), surveyed 
state coordinating and governing boards on their collection and use of postsecondary student-level data. 
Following this, SHEEO identified seven states whose survey responses indicated an exemplary use of data in 
specific subject areas. In-person interviews were conducted by SHEEO agency staff in seven states selected 
for follow-up. In 2015, SHEEO visited the University of North Carolina to discuss teacher preparation data, 
reverse transfer, and database structures. 


In complex political and social environments where the demand for transparency and accountability is on the 
rise, the University of North Carolina (UNC) has succeeded in meeting that demand through its development 
of an integrated, public, postsecondary data system designed to meet the diverse needs of key stakeholders 
including policymakers, institution leaders, faculty, students, and communities. The University of North 
Carolina's new Data Dashboard is the result of the system's commitment to building a culture of data driven 
decision-making and public trust, with an emphasis on strategic planning, collaboration, and continuous 
improvement. This paper highlights the innovative work of the University of North Carolina, including its 
collaboration with the state's 58 community colleges. The system's new Interactive Data Dashboard offers 
other institutions of higher education a model for transforming the nature and application of institutional data. 


UNC is a public university system serving more than 200,000 students annually, with a Board of Governors 
serving as the policymaking body responsible for the oversight and governance of the 17-campus system. 
UNC's campuses are located in cities across the state as illustrated in the map below. Contextual parameters 
for this paper include data systems relative to educator quality and reverse transfer programs. Transparency, 
accountability, and assessment capacity-building to foster data driven decision-making processes were 
universal goals of UNC's work discussed in this paper. 
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Source: https://www.northcarolina.edu/content/our-17-campuses 


1. Appalachian State University, Boone, NC 10. UNC Charlotte, Charlotte, NC 

2. East Carolina University, Greenville, NC 11. UNC Greensboro, Greensboro, NC 

3. Elizabeth City State University, Elizabeth City, NC* 12. UNC Pembroke, Pembroke, NC** 

4. Fayetteville State University, Fayetteville, NC* 13. UNC Wilmington, Wilmington, NC 

5. North Carolina AST State University, Greensboro, NC* 14. UNC School of the Arts, Winston-Salem, NC 

6. North Carolina Central University, Durham, NC* 15. Western Carolina University, Cullowhee, NC 

7. North Carolina State University, Raleigh, NC 16. Winston-Salem State University, Winston-Salem, NC 
8. UNC Ashville, Ashville, NC 17. North Carolina School of Science and Mathematics, 
9. UNC at Chapel Hill, Chapel Hill, NC Durham, NC*** 


* 


Historically Black Colleges and Universities (HBCU) 
** Historically American Indian Liberal Arts University 
*** Public residential high school focused on science, technology, engineering, and math (STEM) 


UNC's drive toward increased accountability, transparency, and a culture of data driven leadership is 
aligned with the Department of Education 2015 executive actions established to promote transparency 
and accountability in higher education to ensure that institutions are effectively serving students and 
families while staying accountable for taxpayer dollars, according to Cecilia Mufoz, director of the 
Whitehouse Domestic Policy Council.’ 


The survey administered to UNC administration as part of SHEEO’s Strong Foundations project provided 
foundational information for this paper. Survey responses revealed that, in 1980, UNC established its 
student unit record system in response to federal civil rights mandates. These data were collected to inform 
policymaking at the system and institution levels, and reports (including critical statistical information) 
were generated for a wide range of purposes and stakeholders. Over the years, UNC has exceeded federal 
data collecting mandates and reporting requirements, underscoring the University’s commitment to 
transparency and accountability. The next section of this paper provides an in-depth examination of UNC's 
approach to accountability and transparency with intentionality around collaboration. 


Informed by survey results, a series of follow-up campus interviews, information retrieved from the UNC 
website and follow-up phone conversations, this paper presents a model for data administration rooted 
in principles of effective change management and collaboration. To further our understanding, campus 
interviews were conducted by SHEEO policy staff with UNC leadership and staff from Academic Affairs, 
Data and Analytics, and Information Technology. 


Cecelia Mufhoz, "Promoting Greater Transparency and Accountability in Higher Education,” 2015. Retrieved August 23, 2016. 
https://www.whitehouse.gov/blog/2015/11/23/promoting-greater-transparency-and-accountability-higher-education 
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ACCOUNTABILITY AND TRANSPARENCY 
THROUGH COLLABORATION 


In 2015, North Carolina passed legislation requiring cross-agency data sharing (House Bill 401-Authorize 
Data Sharing for North Carolina Longitudinal Data System) to enhance meaningful communication around 
common state issues such as education, workforce outcomes, and graduation.’ The legislation underscores 
the North Carolina legislature's mandate to expand the uses and accessibility of data. Although cross-sector 
sharing previously occurred at UNC, a new data dashboard improved its effectiveness and the capacity 
to do so in a very different way, according to Alisa Chapman, vice president for Academic and University 
Programs. Responding to persistent internal and external forces calling for meaningful data for use by multiple 
stakeholders and for multiple purposes, UNC's Data Dashboard represents an innovative and promising 
approach for increasing transparency and institutional accountability, while strengthening a culture of data 
driven decision-making. 


The prototypical UNC Interactive Data Dashboard is the result of a collaboration with SAS, through which the 
goals to promote accountability and transparency were reportedly achieved. Although this paper focuses 
on just two of the eight components of the Data Dashboard, comparable levels of deep collaboration 
were required to achieve the same measure of results as those illustrated in the “Transfer Students” and 
“Educator Quality” components of the Data Dashboard. UNC was well positioned to launch the entire Data 
Dashboard due to extensive collaboration with institutional leadership about building capacity for data 
usage and strategic planning. Furthermore, there is now enhanced capacity for greater responsiveness to 
the general public seeking information. In essence, the data presented in the Dashboard represent efforts 
to be more public and to make data more accessible, according to Chapman. 


The Dashboards are the public face of UNC data, and at the same time that they were being developed 
UNC also embarked on an overhaul of its processes for sending, receiving, and storing institutional data 
into a central Student Data Mart. According to Dan Cohen-Vogel, Associate Vice President for Data and 
Analytics, UNC leadership acknowledged the need for a more future-looking data system, which would 
require significant and sustained investment and necessitate buy-in throughout the 17-campus system. 
The intentional process included a preliminary survey of a wide representation of internal stakeholders 
including the departments of Institutional Research, Information Technology, the Registrar, Admissions, 
and others. Because previous data systems were largely accessible by only a small set of programmers, it 
was important for this initiative to facilitate a “from-the-ground-up development.” It was noted by Cohen- 
Vogel that “we're never going to get away from the day when somebody with skills is needed to access the 
data. It’s just that a lot more people can get at it, there's a lot more transparency about how everything is 
derived and the frequency of collection, and which ones are for reporting instances, and which ones for 
testing purposes. There is just a lot more information about the data and how to use the data now." 


The system building process involved rolling out a baseline code with teams assigned to every constituent 
institution. Teams engaged in planning with campus representatives from each of the surveyed areas, and 
asked, “This is the baseline; how does it actually work in your Banner® or PeopleSoft® system; and where do 
you find this?” The aim was to make sure staff were customizing data for their campuses, applying business 
practices, or changing business practices where necessary. According to multiple UNC representatives, 
the process was reportedly as valuable as the data system itself. Additionally, giving Institutional Research 
units the opportunity to shift resources from creating UNC General Administration files to creating some 


2. North Carolina House Bill 401-Authorize Data Sharing for North Carolina Longitudinal Data System. 
http://www.ncleg.net/Sessions/2015/Bills/House/PDF/H401v2. pdf 
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value added analysis for their university was a huge part of the vision. Other benefits of this system are 
“creating one version of the truth; creating a shared service; just creating a broader access or a broader 
analytical capacity for all of us," according to Cohen-Vogel. 


The high level of collaboration required the various users to ensure the data were accurate by providing 
opportunities to fix errors on the front end of reporting. UNC's process was focused from the beginning 
on developing a sense of shared data governance with common ground rules. It was also built with an 
understanding that it must support important state and federal reporting and other strategic projects, such as 
North Carolina's State Longitudinal Data System (SLDS) and the Reverse Transfer system developed jointly by 
UNC and the state’s community college system. 


Another highlighted benefit of building stronger data integrity included a process where all the people who 
needed to be at the table were at the table (those responsible for the business practice—registration, admission, 
financial aid, etc.). They were the people inputting the data and helping to make sure UNC was pointing at 
the right place to get the intended information. The process and its outcomes were not without challenges, 
but data integrity, systematic data governance, and improved communication between universities and 
community colleges remained the focus, In the short run, this has built stronger communication and data 
quality, and in the longer term it is expected tofree up time to increase analytical capacity, both because it 
reduces some of the historical workload for the institutional research units and because the Data Mart is now 
the foundation upon which analytical and business intelligence tools can sit. 


The Data and Analytics unit staff led efforts to focus on structural issues to standardize data inputs and 
outputs in a way that would make the data more accessible to the public. Development of the Student Data 
Mart began in early 2013 in alignment with the UNC Interactive Data Dashboard. Although the Student Data 
Mart will support dashboards going forward, and currently makes it easier to build interactive tools, one is 
not dependent on the other; they are two separate projects. According to an interviewee, “What we tried 
to do with the Student Data Mart project is make sure we could align the interest of the data owners and 
those who are charged with reporting from it. We devised the data system based on different categories of 
student supports and then met with all of the key areas to ask, ‘Does this definition work with your current 
practice?’” Also noted were the big gains in data quality. The burden has shifted from Data and Analytics to 
the actual data owner—the Office inputting the data. For example, instead of campus institutional research 
units “cleaning” financial aid data, it is pushed back to the source for cleanup (they are the data owners 
and the content area experts). According to Cohen-Vogel, UNC developed extensive documentation and 
a process by which central office teams worked extensively with campus teams to confirm that the items 
they were pulling from Banner or PeopleSoft aligned with system definitions, that business processes 
conformed to the extent it made sense, or that definitions, edits, etc., were informed appropriately by the 
campus processes and enterprise resource planning (ERP) system limitations. For example, they require 
that all completed applications are moved from the external systems into the ERPs (e.g., Banner) and 
that campuses report all completed applications to UNC General Administration. Similarly, all completed 
FAFSAs (Student Information Records [SIRS]) come into the UNC system, not just those associated with 
financial aid awards. 


Perhaps most importantly, according to Cohen-Vogel, UNC does not allow any outside-the-system fixes. 
“If data need to be corrected, it must be done within the ERP system and retransmitted to the central 
database.” UNC personnel believe this saves time by making the process more efficient. Data relationships 
between the campuses and the system have reportedly improved as a result of building the Student 
Data Mart, because it was built collaboratively. An interviewee reported, “It wouldn't matter if we had a 
technically superior design and all that if we didn't have the people to implement it and the buy-in from 
campuses to get excited about giving us good data.” Another interviewee recalls positive feedback from 
institutional research on campuses: “Thank you for pushing the onus on the data back to the person who 
should be responsible for it; thank you for making our registrars clean their course data. They're starting 
to see the value.” 
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Cohen-Vogel recalls the large amount of planning and preparatory work that preceded the initial rollout of 
the new Data Mart system to the campuses, including four to six months of building the baseline code, of 
planning timelines, etc. “It’s an efficiency gain,” according to a member of the Data and Analytics unit. This 
system makes it a more efficient and less manual process and gives us the capacity to efficiently address 
questions of external and internal stakeholders.” In addition to efficiency, transparency, and accountability, 
there Is a political value as well. 


The following discussion examines UNC's innovative dashboards, with emphasis on two data portals— 
Educator Quality and Transfer Students. 


THE UNC DATA DASHBOARD 


Institutions of higher education track a variety of data that augment IPEDS data, such as faculty salaries, 
research expenditures, rankings, research output, and faculty reputation. Such information systems 
are typically anchored and in through an office dedicated to institutional research and assessment, with 
variability in naming, how they operate, and in how they are structured. Regardless, they are commonly 
responsible for collecting, reporting, and providing analytic data for compliance purposes and to enhance 
decision-making. Some data and reports are widely available, including to the public, as evidenced by UNC's 
innovative Interactive Data Dashboards. 


The UNC Data Dashboard (henceforth Data Dashboard) is described on the UNC website as an “interactive, 
online database that gives students, parents, policymakers and taxpayers expanded access to detailed system 
data on selected core measures. This new tool reflects UNC's commitment to increase transparency, better 
track and measure our productivity, efficiency, and impact; and demonstrate greater accountability to the 
people of North Carolina.”* University officials acknowledged the generous support received from SAS® for 
the development of the Data Dashboard.’ 


The Data Dashboard is user-friendly and clearly organized by eight categories of data and information: 


1. Enrollment 5. Degrees 

2. Freshmen Admissions and Performance 6. Educator Quality 
3. Transfer Students 7. Facilities Usage 
4. Research and Development 8. Jobs for NC 


3. University of North Carolina. UNC Data Dashboard. https://www.northcarolina.edu/about-our-system/special-initiatives 


4. University of North Carolina. UNC Infocenter. https://www.northcarolina.edu/infocenter#interactiveData 


© 2016 by the State Higher Education Executive Officers Association (SHEEO) 


(\ SHEEO 


The chart below summarizes each of the aforementioned categories. 


CATEGORY OVERVIEW 


Enrollment Enrollment trends across the 16-campus UNC system (excludes NC School 
of Science and Mathematics). 


Freshmen Admissions and Admissions and performance information about high school graduates pursuing 
Performance degrees at all universities. Graduation rate information. Peer comparisons. 
Transfer Students Trend data on transfer student enrollment, first-year performance, 


and graduation rates. 


Research and Development Trends in research and sponsored programs including award totals for external 
funding as well as the sources of those funds, and information about the number 
of commercial innovations coming out of the University. 


Degrees Data on degrees awarded over the past ten years; data can be sorted by level 
of degree, field of study and demographics. 


Educator Quality Interactive online tool for viewing and analyzing data reflecting UNC progress towards 
the goal of augmenting the quantity and quality of public school educators serving 
North Carolina's students. 


Facilities Usage View classroom and instructional laboratory usage at multiple levels (rooms and 
seats, by day and by hour) including comparisons to norms and historical trends. 


Jobs for North Carolina Through a partnership with the Department of Commerce's Labor and Economic 
Analysis Division (LEAD), UNC tracks employment outcomes of students who stay 
and work in NC. In-depth information is presented through NC's Tool for Online 
Workforce and Education Reporting (NC TOWER), providing post-graduation 
employment, wages and other education outcome details. 


Source: https://www.northcarolina.edu/content/unc-data-dashboard 


The notion of effective data systems has been studied over the years, including by Thomson Reuters (2010), 
which identified five top essentials of model data systems: 

1. Standard Definitions 

2. Broadly Accessible Data 

3. Timely Updates 

4. Multiple Performance Measures 


5. Data Granularity.° 


The data and information contained within the Data Dashboard exhibit these essentials and the Dashboard is 
offered as an exemplar for consideration by any institution seeking to upgrade the effectiveness of its current 
data and reporting systems. UNC's work in this area is particularly impressive given its diversity of institutions 
and the value they ascribe to cross-institutional communication. The next section of this paper highlights the 
two categories of data mentioned above—transfer students and educator quality. 


5. Thomson Reuters. "Finding Meaningful Performance Measures for Higher Education: A Report for Executives.” 
Retrieved August 23, 2016 from http://ip-science.thomsonreuters.com/m/pdfs/exec_report_provosts.pdf. 
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UNC DATA DASHBOARD: TRANSFER STUDENTS AND THE REVERSE TRANSFER PROGRAM 


Policymakers and institution leaders are increasingly interested in performance metrics concerning transfer 
students, defined by UNC as “students entering the reporting institution and known to have previously 
attended a postsecondary institution at the same level (e.g., undergraduate).”° Nationwide, transfer 
students are significant in number although sometimes understudied in policy and research discussions. 
A 2015 report released by the National Student Clearinghouse Research Center highlights the fact that 
one-third of all students switch institutions at least once before earning a degree.’ The report's findings 
highlight the complex nature of higher education and underscore the need for multiple ways of describing 
concepts of student success and institutional effectiveness. The report endorses the practice of collecting 
and analyzing data on outcomes for all students regardless of their enrollment status, including transfer-in 
and transfer-out students, for the benefit of policymakers and students. States and institutions seeking to 
increase college attainment must pay attention to completion data for transfer students. However, Logue 
(2014) asserts that holding institutions accountable for the success of transfer students is challenging in the 
absence of meaningful data. Logue further asserts that “without attending to transfer students’ particular 
circumstances as reflected in their data, without incentivizing colleges to help these students graduate 
and facilitating their credit transfer, increasing the United States’ percentage of young adults with college 
degrees will be far more difficult.”® 


The Data Dashboard collects and disseminates data on transfer students across the system. Data and 
information are contained under the Transfer Students link of the Data Dashboard, providing general 
information on transfer students, transfer enrollment trends, community college transfer trends, transfer 
students’ first-year GPA (grade point average), and graduation rates. The screenshot below from the UNC 
Data Dashboard illustrates the robust nature of UNC's transfer student data. The data are also disaggregated 
by institution.’ 


6. University of North Carolina. From link, click on the tab “About the Dashboard" https://ung4.ondemand.sas.com/ 
SASVisualAnalyticsViewer/quest.jsp?appSwitcherDisabled=true&reportViewOnly=true&reportPath=/UNG/External+Content/ 
Reports&reportName=transfer_enrollment_report 


7. National Student Clearinghouse Research Center. (2016) Snapshot Report: Postsecondary Student Mobility. 
www.nscresearchcenter.org. Retrieved April 11, 2016. 


8. Alexandra Logue, "Wither Thou Goest, | Will Go," Inside Higher Education, November 20, 2014. Retrieved April 11, 2016, from 
https://www.insidehighered.com/views/2014/11/20/essay-importance-transfer-students-increasing-college-completion-rate. 


9. University of North Carolina Transfer Student Data. https://ung4.ondemand.sas.com/SASVisualAnalyticsViewer/guest. 
jsp?appSwitcherDisabled=true&reportViewOnly=true&reportPath=/UNG/External+Content/Reports&reportName=transfer_ 
enrollment_report 
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TRANSFER STUDENT DATA —UNC TOTAL 


eoe oO @ ung4.ondemand.sas.com/SASVisualAnalyticsViewer/quest jsp?appSwitcherDisabled=true&reportViewOnly=true&reportPath=) oO) 4) 7 \g@ 


| Transfer Student Overview | Transfer Enrollment Trends | Community College Transfer Trends | Transfer Student 1st Year GPA | Graduation Rates | About the Dashboard 


(enero | su [| ecsu | ecu Fsu | _Ncaat ecu | ncsu | UNC-CH UNcA uncc | uncs | uNce NCSA uNcw weu wssu 


Where Do Students Transfer From (Fall 2015) ‘Top 20 Community College Transfer Institutions (Fall 2015) 


NC Comm. College System Coastal Carolina CC Mitchell CC Craven CC 
9,373 Fayetteville CC 


57.9% 


Catawba 
Valley CC 


Caldwer Cental Sandhiis 
cc 


Carolina 
cc 


Asheville-Buncombe CC 


Central Piedmont CC 


Rowen-Cebars. jlamance CC 4 


Number of Transfer Students From Each Community College (Fall 2015) 


NC Private 
1,200 


Transfer Institution ‘Transfer Students | 
74% NC Comm. College Total 9373 
Central Pledmont CC 4,005 
Wake CC 969 
Cape Fear CC 601 
Guilford CC 554 
Fayettevile CC 442 
Pitt oC 364 
Forsyth CC 361 
‘Asheville-Buncombe CC 319 
UNC-to-UNC. Durham CC 305 
2 ae a Coastal Carolina CC 241 
Out-of-State Catawba Valley CC 223 
3,478 Rowan-Cabarrus CC 2146 
21.5% ‘Alamance CC 209 
Transferred From cece 187 
'MLNC Comm. College System | Out-of-State ml UNC-to-UNC mi NC Private Davidson County CC 182 


Source: https://ung4.ondemand.sas.com/SASVisualAnalyticsViewer/quest.jsp?appSwitcherDisabled=true&reportViewOnly=true&reportPath=/ 
UNG/External+Content/Reports&reportName=transfer_enrollment_report 
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Transfer student data is also collected to support UNC’s Reverse Transfer Program, which represents 
collaboration between North Carolina's (NC) 58 community colleges and UNC's 16 constituent 
universities (excludes the North Carolina School of Mathematics and Science). Students who transfer to 
a NC university from a participating NC community college are given the opportunity to combine credits 
earned at the university with credit previously earned at the community college to determine if associate 
degree requirements have been met. North Carolina community colleges and UNC institutions began 
participating in the NC Reverse Transfer Program in July 2015. According to a UNC representative, year 
one of the initiative resulted in 665 more associate degrees conferred to students (a 12% increase over one 
year for the community college system). North Carolina community colleges conferred almost 29,000 
degrees in 2014-15, based on most recent IPEDS data. The total through reverse transfer exceeded 2,100 
through the end of 2015, according to Cohen-Vogel. 


The Reverse Transfer Program is aligned with UNC's Student Data Mart, which provides transfer student 
transcript data to the community colleges according to specifications recommended by the community 
colleges. “Here's how we want to see it [data],” according to an interviewee. To that end, actual and agreed 
upon course equivalencies are listed next to the university courses on student transcripts. The program 
is reinforced by a comprehensive articulation agreement, which is a statewide agreement governing the 
transfer of credits between NC community colleges and NC public universities. (North Carolina Community 
Colleges: Comprehensive Articulation Agreement).”” 


. North Carolina Comprehensive Articulation Agreement. http://www.nccommunitycolleges.edu/academic-programs/college- 


transferarticulation-agreements/comprehensive-articulation-agreement-caa. 
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Community colleges run degree audit and missing credit reports each semester. The goal is to identify 
and advise students who have the requisite credits to graduate. In cases where students have attended 
multiple community colleges, the institution that had the most hours for the student gets credit for the 
reverse transfer. In practice, and with student agreement to release transcript data, each of the 16 universities 
sends data to all 58 community colleges to perform degree audits for the associate degree. UNC's Transfer 
Student: Reverse Transfer Program is aligned with the Clearinghouse Reverse Transfer Project as a “major 
step in improving higher education outcomes, which will benefit us as a nation. More students will get the 
degrees they deserve. Community colleges will be recognized for the value they add to education. And 
by granting more degrees, states will be better positioned to attract new business,” according to Walter G. 
Bumphus, president and CEO of the American Association of Community Colleges.” 


UNC DATA DASHBOARD: EDUCATOR QUALITY 


The UNC Board of Governors met in February 2015 and endorsed several strategic recommendations on 
teacher and school leader quality intended to strengthen and focus UNC's educator preparation programs 
(EPPs) with the aim of building a world-class educator workforce to support student success in North Carolina's 
public schools. An educator preparation program (also referred to as a teacher preparation program) refers to 
a state approved course of study. When students complete an approved course of study, they are assumed 
to have met all educational and clinical requirements for certification or licensure to teach at the primary or 
secondary level in the state’s schools. 


In collaboration with SAS® Institute, UNC responded to an explicit major recommendation—the development 
of a public, interactive, web-based tool to help create greater public accountability, transparency, and public 
access for all education stakeholders. The resulting exemplary Educator Quality Data Dashboard provides 
mandated comprehensive and public access to UNC's research data on the performance of educator 
preparation programs across the 15 campuses offering this option of study (UNC School of the Arts and the 
NC School of Science and Mathematics do not offer an EPP). The EPP performance outcomes and indicators 
are provided according to the categories below: 


e Recruitment and Selection (Selection Criteria, Academic Profile Comparisons, and 
Licensure Exam Results); 


e Educator Preparation (Enrollment Trends, Clinical Experiences, and Time to Degree); 


e Performance and Employment (Preparation Pathways, Retention Rates, Recent Graduate 
Surveys, Program Effectiveness Report, Value-Added Models, Teacher Evaluation Ratings, 
Employment Distribution, and Job Placement Rates, School Administration Program Data 
[School Characteristics of NC Principals and Individual Characteristics of NC Principals]; and 


e University-School Partnerships (mapped data on each university-school partnership by 
county and institution).” 


Note: UNC's SAS® EVAAS™ (Education Value-Added Assessment System) is currently under development and reportedly uses test 
scores from a variety of assessments and follows individual students over time to provide reflective value-added reports to assess 
the effectiveness of districts, schools and teachers. This indicator will include EVAAS estimates of teacher effectiveness by institution, 
grade level, and subject area. 


11. Walter G. Bumphus (quoted). National Student Clearinghouse. Reverse Transfer Project. www.reversetransfer.org. 


12. UNC Educator Quality Dashboard. http://eqdashboard.northcarolina.edu 
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The screenshots below from the UNC Data Dashboard illustrate the robust nature of UNC’s Educator Quality 
Dashboard. In these examples, Total Enrollment of Education Majors by Program Level and Average Months 
to Degree are also disaggregated by each institution. 


TOTAL ENROLLMENT OF EDUCATION MAJORS BY PROGRAM LEVEL 


Enroliment of Education Majors by Program Level 


13,218 


Program Level 
{@ Bachelor's 1 Master's 


Education Program Enrollment Over Time 


T T T T T T 
2010 2011 2012 2013 2014 2015 


Source: https://ung4.ondemand.sas.com/SASVisualAnalyticsViewer/guest jsp?reportPath=/UNG/External+Content/ 
Reports&reportName=TQD+Enrollment+Trends&appSwitcherDisabled=true&reportViewOnly=true 


AVERAGE MONTHS TO DEGREE 


Average Months to Degree 


2004-2005 2005-2006 2006-2007 2007-2008 2008-2009 2009-2010 2010-2011 2011-2012 2012-2013 2013-2014 
| | Monms [Years | Terms (Fis) [Temms(Fis..] Hours | > 


Time-to-Degree Trend Data 


Source: https://ung4.ondemand.sas.com/SASVisualAnalyticsViewer/guest.jsp?reportPath=/UNG/ 
External+Content/Reports&reportName=TQD+Time-to-Degree&appSwitcherDisabled=true&report 
ViewOnly=true 
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From the outset of the Data Dashboard project, it was important for academic offices to be engaged. The 
collaboration leading to the Educator Quality Dashboard was led in large part by Alisa Chapman, who 
described a longstanding and productive relationship with the deans of educator preparation programs at 
the 15 constituent universities with EPPs (excludes North Carolina School of the Arts and School of Science 
and Mathematics). In consultation with deans, Chapman started thinking through what educator quality 
outcomes and indicators were needed and for what purposes. Parallel to those conversations, the UNC 
Board of Governors convened a special task force to closely examine teacher preparation and school leader 
preparation. The task force allotted considerable time to visiting constituencies across the state and nation. 
They visited UNC colleges of education. They visited public school representatives in groups and individually. 
They convened forums to get input about particular issues that concerned them. 


Specifically related to educator quality in the state, policymakers and other stakeholders were asking, “What 
are you doing to change the outcomes? How are you going to use research to drive program improvements?” 
According to Chapman, this was a turning point that would require a cultural shift requiring a substantial 
amount of change at the institution level. A major shift included moving from a narrow focus on teacher 
quantity to including teacher quality in the discussions and analyses. North Carolina committed to a process 
of measuring the number of teachers they were producing as well as a process for measuring graduates’ 
effectiveness and impact on North Carolina public schools. As part of the process, UNC established a 
memorandum of understanding (MOU) with the National Council on Teacher Quality (NCTQ), an educator 
preparation research and policy group. The MOU granted NCTQ access to certain EPP-related UNC databases. 


To complement collaborative efforts to prepare quality teachers for North Carolina's public schools, the Office 
of State Human Resources hosts regional personnel administrator meetings that include the State Board of 
Education, dividing the state into eight education regions. The Educator Quality Dashboard provides information 
that can now be used to inform hiring decisions in school districts. Chapman explained that policymakers and 
public school partners are beginning to understand the interactive dashboard and how it can help to inform 
decisions. “We've done a really good job in connecting the dots.” Chapman did acknowledge fear on campuses 
concerning how data gets used, and expressed the need for continued work in this area of communication. 
UNC produces value-added models that control for 36 different co-variants to equalize the playing field. Data 
is aggregated back to educator preparation programs and not individuals. UNC's research on the impact of 
teacher preparation programs on student achievement, measured in terms of their students’ performance on 
state tests of Knowledge and skill, shows how graduates of UNC teacher education programs compare (a) 
with teachers educated at other colleges and universities, both within and outside of North Carolina, (b) with 
teachers who entered teaching via other routes, such as “alternative entry” from other occupations, (c) from 
campus to campus within the UNC system, and (d) from program to program within the system. 


To strengthen transparency and accountability aims, UNC also engaged five institutions to pilot an assessment 
instrument to evaluate teacher candidates (student teachers). Developed by Stanford University, edTPA 
(education teacher performance assessment) is a performance-based, subject-specific assessment and support 
system used by educator preparation programs to emphasize, measure, and support the skills and knowledge 
teachers need to be effective in the classroom. Chapman asserts that edTPA is a valid instrument that shows 
great promise of predictive validity and notes the recommendation from the UNC Board of Governors that all 
institutions have valid and reliable assessment instruments. Although using edTPA, per se, is not mandated, 
Chapman believes edTPA is a way to begin to drive change at a policy and system level. 


When asked if access to more robust, accurate, and more widely accepted data has changed UNC educator 
preparation programs, Chapman expressed optimism about the potential for a positive impact: “The deans 
love this research. They are data driven and focused on outcomes.” 
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SUMMARY 


UNC representatives are aware of the current social and political environments in which higher education 
operates, including a marketplace that is pushing business-intelligence-like global tools for local analysis, like 
SAS®, to analyze higher education outcomes. The success of the UNC Data Dashboard was college driven to 
a large extent and institutional leaders recognized their unique needs for data analysis, in particular, the tools 
that would empower data users to make more effective decisions. There was also system-wide consensus 
and interest in building an infrastructure where data analytic tools would be used consistently across the 58 
community colleges and 16 universities. 


Details about what it would entail to build such an infrastructure and what the benefits and costs would 
be resulted from initial conversations with the Data and Analytics unit, which brought affected personnel 
together at UNC Greensboro for an all-day meeting. “Talking them through what the work would entail, what 
would be involved, what the ups and downs would be, the costs and benefits—this was initiated from within. 
And bringing that up to leaders and saying ‘this is why we need this project,” was essential to the process, 
according to Cohen-Vogel. 


This high level of collaboration and shared responsibility has led to increased institutional effectiveness 
and efficiency. There’s no going back; it would be costly and painful and frustrate everybody to go back, 
according to staff. Ongoing progress is supported with annual meetings that include staff from the Offices 
of the Registrar and Information Technology, to discuss data usage in addition to policy updates or changes 
that affect data. Monthly calls focusing on the Data Dashboard include the Offices of Data and Analytics, 
Information Technology, Financial Aid, and any functional owner who has data coming through the system. 
The goal is to manage the flow of communication. It's a perpetual effort, according to one staff member who 
said, “It's not simply a matter of technology—it’'s also people and culture.” In essence, the success of UNC's 
Data Dashboard is an exemplar of balancing process with a focus on outcomes. 
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